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(54) Control of flashlight in a digital camera 



(57) The invention is based on a method for control- 
ling a flashlight in connection with a digital image sensor 
without a mechanical shutter, wherein the exposure (Rl) 
of the image sensor is performed in sub-areas, such as 
pixel rows or columns, or in another order, by using a 
rolling electronic shutter (10) or the like. Thus ; the ex- 
posure (Rl) of said sub-areas of the image sensor takes 
place at least partly at different times. According to the 
invention, during the time of exposure of one image, /. 
e. substantially all the sub-areas of the image sensor, 
several discrete, successive flashes of the flashlight are 
used to achieve even exposure of the image area, which 



the flashes are timed in such a way that each single flash 
is triggered at a point of time which is common to the 
integration or exposure time (Rl) of as many successive 
sub-areas of the image sensor as possible. According 
to the invention, the successive light flashes are also 
timed in such a way that one and only one flash of the 
flashlight occurs during the integration or exposure time 
(Rl) of each single sub-area. By the present invention, 
the significant advantage is attained that the invention 
minimizes the number of light flashes required for one 
image while it also secures that each sub-area of the 
sensor to be exposed at a different time is illuminated 
as evenly as possible. 



Rl = 12 RP 



CM 
< 

CM 

CO 
CO 
CM 



CL 
LU 



Row 12- 



Row 0 



g^^-RowH 



Row 23- 




1 st flash 2nd flash 3rd flash 



time 



Fig. 2 



Printed by Jouve, 75001 PARIS (FR) 



BNSDOCID: <EP 1 26321 5A2_L> 



1 



EP1 263 215 A2 



2 



Description 

[0001] The invention relates to a method for control- 
ling the operation of a flashlight in connection with an 
electronic image sensor operating without a separate 5 
external shutter, as set forth in the preamble of the ap- 
pended claim 1 . 

[0002] Digital image sensors based on semiconduc- 
tor technology, such as matrix sensors based on charge 
coupled devices (CCD) and complementary metal oxide 
semiconductor (CMOS) technology, now make it possi- 
ble to provide a large variety of devices with built-in im- 
aging functions. Such image sensors are used e.g. in 
digital video and still cameras intended for consumers, 
as well as in various camera devices connected to com- 
puters, such as so-called network cameras. Thanks to 
their high degree of integration, compact size and low 
power consumption, particularly CMOS sensors are al- 
so very suitable for use in small-size portable devices, 
such as mobile stations and so-called personal data as- 
sistants (PDA). 

[0003] Particularly in a situation in which a digital cam- 
era, i.e. a digital image sensor, is implemented in a de- 
vice which is small in size (portable) and/or inexpensive 
in its sale price, it is important to implement the camera 
function with structures which are as simple as possible. 
The aim is thus to minimize the space taken by the cam- 
era in the device as well as the power consumption and 
the total manufacturing costs of the device. 
[0004] One way to simplify the structure of the digital 
camera is to eliminate a separate, typically mechanically 
operated shutter which is used in front of the sensor ma- 
trix to control the exposure time. In such shutter-free dig- 
ital cameras, the exposure time is controlled electroni- 
cally by controlling the functions of the sensor matrix. 
[0005] In the following, we shall briefly describe the 
operation of a CMOS sensor based on the use of an 
electronic shutter, as well as the problems caused by 
the use of the electronic shutter in practice. 
[0006] To put it simply, the CMOS image sensor con- 
sists of a matrix of photosensitive pixels. When light 
strikes a single pixel, the pixel is charged with an electric 
charge which is proportional to the amount of incoming 
light and is further stored in or in connection with said 
pixel. To read the pixel value, the charge is converted 
by means of a charge amplifier to a voltage which is fur- 
ther conducted via an analog-to-digital (AD) conversion 
out of the image sensor. 

[0007] The exposure time of a single pixel in the 
CMOS sensor consists of the time during which the pixel 
is allowed to integrate the electric charge formed by this 
incoming light. The integration or exposure time starts 
at the point of time when the previous charge contained 
in the pixel is first adjusted to zero by a reset function, 
and ends when the pixel charge is read by a sample 
function. 

[0008] To achieve the best possible image quality, all 
the pixels of the image sensor should be exposed pre- 



cisely at the same time. In other words, the above-de- 
scribed operations of resetting, integration and reading 
should be performed simultaneously for all the pixels in 
the matrix. However, this will result in a very complex 
structure of the sensor. Furthermore, the transfer of im- 
age information in serial and digital form out of the sen- 
sor circuit should thus be performed by using a consid- 
erably large bandwidth. 

[0009] Thus, for the above-mentioned reasons, the 
solution commonly used is to process a CMOS matrix 
sensor row by row, i.e. to perform the operations of re- 
setting, integration and reading for one pixel row of the 
matrix at a time. This makes the sensor structure con- 
siderably simpler, and th us also the transfer of the image 
information out of the sensor circuit will take place nat- 
urally in serial form, row by row, wherein the require- 
ments set for the image information transfer rate are 
easier. 

[0010] However, the row-by-row processing has the 
drawback that the different rows ofthe matrix sensorare 
now exposed at slightly different times. Figure 1 shows, 
in principle, the row-by-row processing of the image 
sensor and its effect on the exposure of the different 
rows of the sensor. 

[001 1] To start the integration or exposure time of the 
rows, the rows are reset one by one with a reset function 
(R). To keep the exposure times of the rows mutually 
equal, the resetting (R) of successive rows takes place 
at the same rate at which the rows will be read with the 
sample function (S) at a later stage. The period of time 
between the resetting operations (R) of two successive 
rows is called row processing time (RP). The minimum 
value for the row processing time (RP) is determined by 
the rate at which the sensor circuit can transfer image 
information out of the circuit. Consequently, the row 
processing time (RP) also indicates the time which is 
taken between the sample operations (S) of two succes- 
sive rows. 

[0012] The row integration or exposure time (Rl) can 
now be formed of suitable multiples of the row process- 
ing time (RP) in such a way that the exposure time (Rl) 
is kept equal for all the rows. In the example of Fig. 1 , 
the time selected for the exposure time is 10 x the 
processing time (RP) After the first row (row 0) has in- 
tegrated light for said exposure time (Rl), the system 
contained in the circuit will read and convert the image 
information of the pixels of said row into digital format 
and output it from the circuit. After this, the image infor- 
mation contained in the next rows in the order will be 
read and outputted in a corresponding way. 
[0013] As shown in Fig. 1, image information will not 
be outputted from the sensor circuit before the point of 
time T 2 , because the integration time (Rl) of the first row 
0 has not expired yet. It can be seen from Fig. 1 that 
during the period between the points of time T-j and T 2 , 
there is a period which is common to the exposure of 
the rows 0 to 4 but is still shorter than the total exposure 
time (Rl) of a single row. However, for example row 0 
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and row 1 1 are exposed at a totally different time. Con- 
sequently, the difference in the time of exposure is great- 
est between the first row 0 and the last row n-1 of the 
sensor. 

[0014] Consequently, the exposure/integration times 5 
(Rl) of adjacent rows of the image sensor, to be proc- 
essed one after the other, are partly overlapping, but the 
exposure/integration takes place clearly at different 
times in rows which are far from each other, for example 
at the upper and lower edges of the sensor. 
[0015] Row-by-row processing of the image area in 
the above-described manner is known from prior art as 
a rolling electronic shutter or as a rolling window shutter. 
[0016] When the object to be imaged is substantially 
immovable in relation to the camera or in a slow motion 
in relation to the processing time of the whole image ar- 
ea (all the rows), and when the lighting is constant with 
respect to time, the rolling electronic shutter will not 
cause considerable harm to the imaging and to the im- 
age quality. 

[0017] However, in a situation in which a flashlight is 
used for illuminating the object during the imaging, con- 
siderable problems will be caused to the image quality 
by the exposure of the sensor rows at different times. 
The reason for this is that because of the short flash time 
specific to the flashlight, the illumination produced by 
the flashlight will significantly change during the time 
when the whole image area of the sensor is exposed/ 
processed. 

[0018] For example, the duration of a flash in flash- 
lights based on a discharge tube, used in pocket cam- 
eras or the like, typically varies from some tens of mi- 
croseconds to some hundreds of microseconds. Corre- 
spondingly, in a CMOS sensor with VGA resolution (640 
x 480 pixels), to be processed row by row, the process- 
ing of the whole image area typically takes several tens 
of milliseconds when a rolling electronic shutter is used. 
Now, as the flash of the flashlight is considerably shorter 
than the processing of the whole image area, this will 
cause that different rows of the sensor will be exposed 
in significantly different ways when the flashlight is used, 
and therefore, the quality of the images taken with the 
flashlight is impaired. 

[001 9] It is the primary aim of the present invention to 
disclose a method forcontrolling the operation of a flash- 
light in connection with a digital image sensor which is 
to be processed row by row, column by column or an- 
other sub-area at a time and which uses a rolling elec- 
tronic shutter or the like and thereby operates without a 
separate external shutter. It is a particular aim of the in- 
vention to avoid the above-presented problems which 
impair the image quality due to the disparity between 
the duration of the flash of the flashlight (light pulse) and 
the time required for processing the whole image area 
of the sensor. 

[0020] To attain this purpose, the method according 
to the invention is primarily characterized in what will be 
presented in the characterizing part of the independent 



claim 1 . 

[0021 ] The other dependent claims present some pre- 
ferred embodiments of the invention. 
[0022] The invention is based on the idea that during 
the time required for exposing one image, several sep- 
arate, successive light flashes are used instead of a sin- 
gle light flash, to balance the exposure of different sub- 
areas of the image sensor to be processed at different 
times and thus integrating the charge at different times. 
[0023] The basic idea of the invention is that the flash- 
es of the flash unit to be used for a single image are 
timed so that each single flash of the flash unit is per- 
formed at a point of time which is common to the inte- 
gration or exposure time (Rl) of as many successive 
sub-areas of the sensor as possible, such as rows, to 
be exposed at a time. Further the successive flashes 
are also timed so that one and only one lightflash occurs 
during the exposure time of each said sub-area. 
[0024] The significant advantage is attained by the 
present invention that it minimizes the number of light 
flashes required for one image while it also secures that 
the object to be imaged is illuminated as evenly as pos- 
sible during the exposure (Rl) of each sub-area of the 
sensor. The minimization of the number of light flashes 
is very important particularly in portable and/or other 
battery-operated devices in which the total power con- 
sumption of the flash should be kept as low as possible 
to save the capacity of the batteries and/or to minimize 
their size. 

[0025] By using the method of the invention, a very 
high image quality is also achieved, thanks to the even 
exposure of the image area. Furthermore, in the case 
of a moving object to be imaged, the invention effectively 
prevents blurring of the image caused by the movement 
of the object, in a way to be described more closely here- 
inbelow. 

[0026] In an advantageous embodiment of the inven- 
tion, in the case of a matrix sensor to be processed row 
by row, the number of flashes required for one image is 
determined by means of the ratio between the row 
processing time (RP) of the matrix sensor and the inte- 
gration or exposure time (Rl) used in the shooting. The 
ratio RI/RP indicates the number of successive rows 
which can be covered by one flash of the flash unit. 
[0027] The flashlight control according to the inven- 
tion can be used in normal camera shooting with a flash- 
light under dark or dim conditions, or also in producing 
a fill-in flash under illuminated conditions. 
[0028] Further, the invention can also be applied in 
the storage of so-called super resolution images. In su- 
per resolution images, several successive images in 
which the object is slightly transferred with respect to 
the image area are combined by means of image 
processing into a single new image. The resolution of 
the combined image is better than the pixel resolution 
of the original images. 

[0029] The invention is preferably applied particularly 
in connection with CMOS image sensors, to produce a 
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simple, compact digital camera with a low power con- 
sumption. Such a digital camera is suitable to be placed 
e.g. in a mobile station or in another corresponding port- 
able personal device. 

[0030] The following, more detailed description of the 
invention with examples will more clearly illustrate, for 
anyone skilled in the art, advantageous embodiments 
of the invention as well as advantages to be achieved 
with the invention in relation to prior art. 
[0031] It should be noted that although, in the follow- 
ing examples, the invention is primarily described in 
connection with a matrix sensorto be exposed and read 
in pixel rows, it can also be used in connection with sen- 
sors to be processed in columns or in sub-areas in an- 
other way. 

[0032] I n the following, the invention will be described 
in more detail with reference to the appended drawings, 
in which 

Fig. 1 illustrates, in principle, how the exposure time 
of a matrix sensor is controlled when using a 
rolling electronic shutter known from prior art, 

Fig. 2 shows, in principle, how the flashes of a flash- 
light according to the invention are timed in 
connection with a matrix sensor to be proc- 
essed and exposed row by row as shown in 
Fig. 1, 

Fig. 3 shows the typical time behaviour of the flash 
of a flashlight in a commercially available dig- 
ital camera, and 

Fig. 4 illustrates, in principle, how the exposure time 
of a matrix sensor is controlled when using an 
electronic shutter function known from prior 
art. 

[0033] The operation of a rolling electronic shutter 
(10), shown in principle in Fig. 1 , has already been dis- 
cussed above in connection with the description of prior 
art. 

[0034] Figure 2 shows, in principle, how the flashes 
of a flashlight of the invention are timed in connection 
with a matrix sensor to be processed and exposed row 
by row according to Fig. 1 or in a way substantially cor- 
responding to Fig. 1. 

[0035] According to Fig. 2, the start of the integration 
or exposure time (Rl) of each pixel row in the sensor is 
delayed from the start of the exposure time (Rl) of the 
preceding row by the time of the reset function (R), i.e. 
by the row processing time (RP). For this reason, the 
exposure times of successive rows are shifted with re- 
spect to each other as shown in Fig. 2. 
[0036] According to the invention, the flashes of the 
flashlight to be used for one image are timed in such a 
way that each single flash of the flashlight is triggered 
at a point of time which is common to the integration or 



exposure time (Rl) of as many successive rows as pos- 
sible. In the case of Fig. 2, the first flash is thus timed to 
the point of time marked with the first vertical line, where- 
in the flash affects the exposure of the rows 0 to 11 , but 

5 not the exposure of the other rows of the sensor. The 
next flash of the flashlight is timed to the point of the time 
marked with the second vertical line, wherein the sec- 
ond flash only affects the exposure of the rows 12 to 23. 
Thus, the second flash does not affect the exposure of 

10 the preceding rows 0 to 11 nor the exposure of the fol- 
lowing rows, starting from the row 24. In a corresponding 
manner, the third flash only exposes the rows 24 to 31 . 
[0037] According to the invention, successive flashes 
. are thus timed so that there is one and only one liqht 

15 flash during the exposure time of each pixel row 0 x 1 
This is important so that the quantity of light proauc^d 
by the flash at each pixel row remains substantially 
equal. At the same time, the minimum number of flashes 
required for the exposure of one image is determined, 

20 wherein the total power consumption of the flash is also 
minimized. 

[0038] The exposure of each pixel in a single pixel row 
starts simultaneously after the reset function (R) to be 
performed for said row and ends simultaneously in the 

25 reading of the pixel values by the sample function (S). 
Consequently, wi +H respect to the quantity of light re- 
ceived by the pixeis in a single pixel row, it is not signif- 
icant at which point of the integration or exposure time 
(Rl) of said pixel row the flash occurs. 

30 [0039] In an advantageous embodiment of the inven- 
tion, the number of flashes required per one image is 
determined by means of the ratio between the row 
processing time (RP) of the matrix sensor and the inte- 
gration or exposure time (Rl) used in the shooting. Said 

35 ratio RI/RP indicates the number of successive rows in 
the matrix sensor which can always be covered by one 
pulse of the flashlight. In the example shown in Fig. 2, 
Rl - 12 RP and thus RI/RP = 12. Thus, 12 successive 
rows can always be covered by means of one pulse. In 

40 the example shown in Fig. 2, where the total number of 
pixel rows is 32, 3 separate flashlight pulses will be re- 
quired to cover these 32 rows. Correspondingly, in the 
situation shown in Fig. 2, Rl = 10 RP, wherein it would 
always be possible to cover 1 0 rows at a time by means 

45 of one flashlight pulse. 

[0040] When shooting in the dark or in dim light, the 
exposure time (Rl) can be selected to be relatively long, 
because under these conditions, there is no significant 
light other than the flashlight coming into the camera 

so sensor. Thus, one light flash can be used to cover sev- 
eral pixel rows of the sensor at a time; that is, fewer 
flashes are required for the whole image area. 
[0041] Correspondingly, under bright lighting condi- 
tions, the exposure time (Rl) is selected to be shorter to 

55 prevent overexposure of the pixel rows because of the 
prevailing light. Thus, as the RI/RP ratio is reduced. ■ > e 
number of flashes required to expose the whole irru- 
area will increase. 



4 



BNSDOCID: <EP 



126321 5A2_I_> 



7 



EP1 263 215 A2 



8 



[0042] For the sake of illustration, Fig. 3 shows the 
behaviour of the light flash (intensity of the light pulse) 
of a commercially available pocket-type digital camera 
(Canon QV-770) as a function of time. The graph of Fig. 
3 can be considered to represent the typical operation 
of a flashlight based on the use of a flash tube built in a 
pocket-type camera or the like, the flashlight having a 
sufficient output capacity to be used in connection with 
conventional matrix sensors, such as CMOS sensors, 
under normal shooting conditions. 
[0043] In Fig. 3, each square interval of the horizontal 
axis represents a time interval of 1 00 microseconds, and 
the vertical axis represents the intensity of the lightflash 
in a relative linear scale. Figure 3 indicates that the du- 
ration of the intensity maximum of said light flash is in 
the order of a few hundred microseconds. 
[0044] For comparison, it can be stated that in a 
CMOS sensor with VGA resolution (640 x 480 pixels), 
to be processed (reset-integration-reading) row by row, 
the processing of the whole image area typically takes 
several tens milliseconds. In atypical CMOS sensor, the 
row processing time can be e.g. about 3500 clock cy- 
cles, which corresponds to 140 microseconds if the 
clock cycle is 40 nanoseconds. The processing of the 
whole image, i.e. 480 rows, will thus take 67.2 millisec- 
onds. 

[0045] From prior art, it is known to use a sensor that 
is processed and exposed electronically row by row in 
a particular shutter mode. This principle is illustrated in 
Fig. 4. The following is a brief description of the opera- 
tion of said shutter mode and differences between it and 
the present invention. 

[0046] In the shutter mode of prior art, the principle is 
that all the pixel rows of the image sensor are set to zero 
by a quick reset function to start the integration time 
(INT) for all the rows as simultaneously as possible. Af- 
ter the integration time (INT) of even the last row has 
started, an external mechanical shutter is opened at a 
point of time T on to start the exposure of all the rows 
simultaneously. At the point of time T off , the external 
shutter is closed, correspondingly, to terminate the ex- 
posure of all the rows simultaneously. Now, the expo- 
sure time for each sensor row thus becomes the open 
time T off -T on of the external shutter. When a single light 
flash is placed between the moments T on and T off , the 
light flash will have the same effect on all the pixel rows. 
After the external shutter has been shut, the pixel rows 
can be read by a sample operation at a rate which is 
limited by the rate of transfer of image information from 
said sensor circuit and which rate of transfer is thus typ- 
ically significantly slower than said quick reset function. 
As the quick reset function and the sample function per 
one row have unequal lengths in time, this will result in 
different integration times (INT) of different pixel rows. 
However, when a separate external shutter is used, this 
will not affect the image quality, because when the shut- 
ter is shut, no light will enter the sensor and the pixels 
of the sensor will thus not be charged. 



[0047] When imaging in the dark or in a sufficiently 
dim light, said shutter mode of the sensor can be used 
in a known manner in connection with a flashlight also 
without a separate external shutter in the following way. 

5 [0048] According to Fig. 4, the flash of the flashlight 
illuminating the object is placed in the same way as if 
there were a separate external shutter used in the cam- 
era. If the light entering the sensor primarily originates 
from the flash only ; all the rows of the sensor will be uni- 

10 formly exposed even if there is no external shutter in 
use. However, if there is also other ambient light coming 
into the sensor, the last rows of the sensor will thus be 
exposed more than the first rows, which may, depending 
on the situation, result in for example the last pixel rows 

15 being saturated, i.e. overexposed. If there is ambient 
light present, the last rows of the sensor will be unevenly 
exposed due to the integration times of different lengths. 
Furthermore, in the case of a moving object, the image 
will also be blurred more at the last rows with the longer 

20 exposure time than at the first rows with the shorter ex- 
posure time. 

[0049] In the solution according to the invention, the 
above-described problems do not occur in this way, be- 
cause the exposure times (Rl) of all the rows are pref- 

25 erably arranged to have equal lengths. For example, in 
the case of a mobile object, this means that the image 
parts corresponding to different flashes of the flashlight 
are not differently blurred in different image parts (differ- 
ent pixel rows) because of the movement, but they are 

30 only shifted with respect to each other. Such a shift is 
significantly easier to correct afterwards by image 
processing methods than artefacts caused by different 
exposure times of different parts in the image. 
[0050] According to the invention, successive flashes 

35 synchronized with the operation of the image sensor can 
be triggered in a suitable way by using one or several 
flash units. 

[0051] As the flashlight is based on an arc discharge 
produced in a conventional flash tube, the electric ener- 

40 gy required for successive flashes (arc discharges) can 
be obtained from a single common capacitor with a suf- 
ficiently high capacity, or from another corresponding 
energy storage which is capable of instantaneously pro- 
viding the power required for the arc discharge. One 

45 flash tube can also be supplied by several discrete ca- 
pacitors or the like, each capacitor being always 
charged with, for example, the energy required for one 
flash. 

[0052] Furthermore, the method according to the in- 
50 vention can be implemented by using several discrete 
flash units, each unit comprising a separate flash tube 
and an energy storage intended for it, or other corre- 
sponding components. The discrete flash units can be 
arranged to trigger flashes which are, according to the 
55 invention, timed successively in such a way that each 
flash unit flashes only once per each image. It is also 
possible to use two or more flash units together so that 
they flash alternately in such a way that each flash unit 
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flashes morethan once per one image. Using flash units 
flashing in an alternating manner provides the advan- 
tage that a single unit does not need to be capable of 
successive flashes at the same rate as when only one 
unit is used. 

[0053] Instead of using a flash tube based on arc dis- 
charge, a discharge lamp or other corresponding ther- 
mal light sources, it is also possible to use non-thermal 
light sources to produce light pulses. Such non-thermal 
light sources include, for example, light emitting diodes 
(LED), particularly so-called super bright light emitting 
diodes. It is known as such that considerably higher lu- 
minosities can be instantaneously outputted by pulsing 
than by continuous operation of light emitting diodes. 
The effect of the relatively narrow spectrum band emit- 
ted by the light emitting diodes on the colour balance 
can be compensated by using several light emitting di- 
odes of different colours simultaneously for the illumina- 
tion. 

[0054] The timing of the successive light pulses, syn- 
chronized with the operation of the light sensor accord- 
ing to the invention, can be implemented in the system 
controlling the operation of the image sensor either by 
software or by using logic circuits or other hardware op- 
erations. The integration or exposure time (Rl) may be 
determined, for example, on the basis of a signal from 
a measuring means measuring the external luminosity, 
wherein the ratio RI/RP between the exposure time (Rl) 
and the processing time (RP) of a row, column or anoth- 
er partial area of said matrix sensor is used to determine 
the required number of flashes as well as their points of 
time during the exposure of image. On the basis of this 
information, one or more flash units are controlled to 
produce the flashes at the right time. 
[0055] The information about the number of flashes 
used for one image and about the exact points of time 
of the flashes can be further utilized in the post-process- 
ing of the image produced by the image sensor. Said 
post-processing is preferably performed in connection 
with the digital camera function in the same device im- 
mediately after taking the picture. 
[0056] Post-processing can be used for example for 
correcting a shift between image parts corresponding to 
different flashes, which shift is caused by the movement 
of the object to be imaged during the time between suc- 
cessive flashes of the flashlight. In the correction, the 
image sub-areas to be transferred in relation to each 
other are determined in such a way that one area to be 
shifted substantially as a whole in relation to the other 
sub-areas always consists of the part of the image area 
stored (exposed) during one flash. By using various im- 
age processing methods, such as movement estima- 
tion, during the post-processing, said different parts of 
the image area can be aligned with each other on the 
pixel level in such a way that the effect of shifts caused 
by movement of the image is eliminated or significantly 
reduced in the corrected image. 
[0057] Post-processing of the image, which utilizes 



the number and exact points of time of light flashes, can 
also be applied in the case of several images stored in 
succession. One possible use of the invention is the 
storage of so-called super-resolution images. In super- 
resolution images, several successive images in which 
the object is slightly shifted with respect to the image 
area are combined by means of image processing into 
a single new image. The resolution of the combined im- 
age can thus be made better than the pixel resolution of 
the original images in a way known as such. 
[0058] When forming super-resolution images by us- 
ing flashlight timing according to the invention, several 
images are stored in succession, in which the different 
image parts corresponding to single light flashes are 
now imaged in a good quality manner under similar light- 
ing conditions and without an artefact caused by the 
movement of the object. The good quality partial images 
can be combined in the above-described manner into 
successive good quality images. These images can fur- 
ther be combined, by using image processing methods 
known as such, into a super-resolution image having a 
better pixel resolution than a single image. The single 
images can be considered to represent the object to be 
imaged from a slightly different camera angle, wherein 
by combining information contained in several images 
it is possible to achieve an image whose information 
content exceeds the information content of a single im- 
age. 

[0059] The combination of single images taken of an 
object in succession from slightly different camera an- 
gles and/or in different positions of the object is prior art 
known as such. The invention makes it possible to use 
a flashlight in connection with taking super-resolution 
pictures, which improves the definition of the image by 
eliminating artefacts which are otherwise due to long ex- 
posure time and movement. Super-resolution imaging 
can be used, for example, in the conversion of paper 
documents into electrical format with a good pixel reso- 
lution. 

[0060] The invention has been described above by 
using a few selected examples. However, it is naturally 
obvious that the above-presented examples must not 
be interpreted as the only possible embodiments of the 
invention, but the embodiments of the invention can be 
freely varied within the scope of the inventive features 
presented in the claims below. 



Claims 

1 . A method for controlling a flashlight used for illumi- 
nating an object to be imaged in connection with an 
electronic image sensor based on a 2-dimensional 
pixel matrix, or the like, the exposure (Rl) of the im- 
age sensor being performed in sub-areas, for ex- 
ample pixel rows or columns, or in another order, 
by using a rolling electronic shutter (1 0) or the like, 
wherein the exposure (Rl) of said sub-areas of the 
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image sensor takes place at least partly at different 
times, characterized in that during the time of ex- 
posure of one image, i.e. substantially all the sub- 
areas of the image sensor, 

— several discrete, successive flashes of a flash- 
light are used to achieve even exposure of the 
image area, which flashes are 

— timed in such a way that each single flash is 
performed at a point of time which is common 
to the integration or exposure time (Rl) of as 
many successive sub-areas of the image sen- 
sor as possible, and that the successive flashes 
of the flashlight are 

— also timed in such a way that one and only one 
flash of the flashlight occurs during the integra- 
tion or exposure time (Rl) of each single sub- 
area. 

2. The method according to claim 1 , characterized in 

that the control of the flashlight is applied in con- 
nection with an image sensor to be exposed pixel 
row by pixel row, wherein the number of pixel rows 
to be simultaneously covered by one light flash is 
determined from the ratio between the integration 
or exposure time (Rl) of a single pixel row and the 
processing time (RP) of a single pixel row, which 
pixel row processing time (RP) indicates the time 
taken for a pixel row reset function (R) and/or read 
function (S), and which exposure time (Rl) and 
processing time (RP) are substantially equal for all 
pixel rows. 

3. The method according to claim 1 , characterized in 

that the control of the flashlight is applied in con- 
nection with an image sensor to be exposed pixel 
column by pixel column, wherein the number of pix- 
el columns to be simultaneously covered by one 
light flash is determined from the ratio between the 
integration or exposure time (Rl) of a single pixel 
column and the processing time (RP) of a single pix- 
el column, which pixel column processing time (RP) 
indicates the time taken for a pixel column reset 
function (R) and/or read function (S), and which ex- 
posure time (Rl) and processing time (RP) are sub- 
stantially equal for all pixel columns. 

4. The method according to any of the preceding 
claims, characterized in that the control of the 
flashlight is applied in connection with an image 
sensor based on CMOS technology. 

5. The method according to any of the preceding 
claims, characterized in that successive flashes 
of the flashlight, required for one image, are trig- 
gered by using one or several discrete flash units. 

6. The method according to claim 5, characterized in 



that the successive flashes of the flashlight, re- 
quired per one image, are triggered as alternating 
flashes of two or more flash units. 

5 7. The method according to claim 5 or 6 ; character- 
ized in that a single flash unit is based on the use 
of a flash tube, a discharge lamp or another corre- 
sponding thermal light source. 

10 8. The method according to claim 5 or 6 ; character- 
ized in that a single flash unit is based on the use 
of a light emitting diode or another corresponding 
non-thermal light source. 

15 9. The method according to any of the preceding 
claims, characterized In that the control of the 
flashlight is applied for producing a fill-in flash in 
camera shooting under illuminated conditions. 

20 10. The method according to any of the preceding 
claims, characterized in that an image taken by 
using the flashlight and stored in electrical format is 
subjected to post-processing to improve the image 
quality by utilizing information about the size of im- 
25 age sub-areas corresponding to successive sepa- 
rate flashes of the flashlight. 

11. The method according to claim 10, characterized 
in that the post-processing is used to correct a shift 

30 caused between said sub-areas due to the move- 
ment of the object to be imaged. 

12. The method according to claim 10 or 11 , charac- 
terized in that two or more images taken in suc- 

35 cession are subjected to post-processing by com- 

bining them into a single super-resolution image by 
image processing. 

13. The method according to any of the preceding 
40 claims, characterized in that the flashlight control 

is applied in a digital camera function implemented 
in or in connection with a portable, battery-operated 
device. 

45 14. The method according to claim 13, characterized 
in that the flashlight control is applied in a mobile 
station or the like. 
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(54) Control of flashlight in a digital camera 

(57) The invention is based on a method for controlling a flashlight in connection with a digital image sensor without 
a mechanical shutter, wherein the exposure (Rl) of the image sensor is performed in sub-areas, such as pixel rows or 
columns, or in another order, by using a rolling electronic shutter (10) or the like. Thus, the exposure (Rl) of said sub- 
areas of the image sensor takes place at least partly at different times. According to the invention, during the time of 
exposure of one image, i.e. substantially all the sub-areas of the image sensor, several discrete, successive flashes of 
the flashlight are used to achieve even exposure of the image area, which the flashes are timed in such a way that each 
single flash is triggered at a point of time which is common to the integration or exposure time (Rl) of as many successive 
sub-areas of the image sensor as possible. According to the invention, the successive light flashes are also timed in 
such a way that one and only one flash of the flashlight occurs during the integration or exposure time (Rl) of each single 
sub-area. By the present invention, the significant advantage is attained that the invention minimizes the number of light 
flashes required for one image while it also secures that each sub-area of the sensor to be exposed at a different time 
is illuminated as evenly as possible. 
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